The present invention relates to an improved motion compensated generation of new image data. In particular, the present invention relates to a method for generating image data based on motion compensated data stemming from other images, wherein the other images comprise separate image areas.
Motion compensation is employed in an increasing number of applications, in particular in digital signal processing of up-to-date television receivers. Specifically, modern television receivers perform a frame-rate conversion, especially in form of a motion compensated up-conversion, in order to increase the picture quality of the reproduced images. Motion compensated up-conversion is performed, for instance, for video sequences having a field or frame rate of 50 Hz to higher frequencies like 60 Hz, 66.67 Hz, 75 Hz, 100 Hz, etc. While a 50 Hz input signal frequency mainly applies to television signals broadcast in accordance with PAL or SECAM standards, NTSC based video signals have a field rate of 60 Hz. A 60 Hz input video signal maybe up-converted to higher frequencies like 72 Hz, 80 Hz, 90 Hz, 120 Hz, etc.
Frame rate conversion algorithms require the generation of intermediate images which reflect the video content at temporal positions different from that of the input video sequence of, for instance, 50 Hz or 60 Hz. During interpolation of image data of the input images, the motion of moving objects has to be taken into account in order to appropriately reflect changes of the image content caused by object motion.
The motion to be used during interpolation of the image data is determined based on the two closest images of the input video sequence. Generally motion estimation is performed on a block basis. In order to avoid visible block structures in the motion compensated images, the motion vectors determined on a block basis are preferably assigned to each individual pixel wherein the resulting field of motion vectors is subjected to filtering in order to smoothen sharp transitions.
In accordance with the motion vector assigned to a particular image position, motion compensation shifts the image data of the image position referenced by the motion vector. The amount of shifting image data of the referenced images does not only depend on the length of the received motion vector but further depends on the relative temporal position of the new image to be generated between the previous and subsequent images.
When generating new image data from two images, preferably the previous and subsequent images, the motion compensation algorithm references image data in the previous and the subsequent images. While the image data from the previous image is shifted forward in the motion vector's direction, the image data from the subsequent image is shifted backwards. The amount of forward and backward shift is determined in accordance with the temporal position of the new image with respect to the referenced images. While the shift of the forward shift is executed in accordance with a calculated fraction of the motion vector's length, the backward shift is performed with the complementary negative fraction of the motion vector. The referenced pixel values at each pixel position are interpolated. The interpolation can also be based on a weighing of the image data from the referenced images based on the temporal position with respect to the newly generated image.
The conventional approach for performing a motion estimation is illustrated in FIG. 1A. From an input sequence 101-113, image data of two adjacent images is used in order to generate motion compensated new image data 130. The newly generated image data 130 reflects the temporal position of moving object 100 within the image sequence in accordance with the temporal position of the new image data 130. As can be seen from FIG. 1A, the position of the moving object 100 in the new image 130 is in-between the positions of images 102, 103 employed for motion compensation. Thus, image 130 correctly reflects the motion phase of the moving object 100.
Each image of the input image sequence 101-113 includes of two separate image areas. Specifically, a ticker window 123 is overlaid on a background image 122. Such a separate image portion is becoming increasingly popular in broadcast television signals. The overlaid ticker window is usually in the form of a text banner inserted at the bottom area of the video image. The text banner displays service information provided by the broadcaster. The displayed service information may either relate to the displayed image content related to program information or to a particular news service such as stock exchange rates. The individual position for inserting a ticker window into the video image may differ depending on the broadcaster or the broadcaster's country.
As illustrated in FIG. 1A, the text displayed in the ticker window 123 is shifted leftwards such that a continuous stream of text moving with a predefined speed is passing through the display screen.
Although, the separate image portions 123 and the background image 122 relate to a different image content, motion compensation provides an accurate generation of new image data for both image portions.
In order to detect scene changes 120 in an input video signal sequence 101-113, the signal processing of the present invention for motion compensated interpolation performs a scene change detection. The scene change detector detects abrupt image content changes between the images involved in the generation of a new image. Details for implementing an image scene detection are known, for instance, from EP-A-0 780 776.
If a scene change 120 is detected between two images 103, 111 involved, the motion compensated interpolation is inhibited in order to prevent the introduction of artefacts into the interpolation image 140. For this purpose, the motion compensation algorithm is replaced by another predefined interpolation algorithm. Preferably the image 140 is generated by only referring to a single one of the two images 103, 111 which would have been referenced by motion compensation. Preferably, a new image data 140 is generated based on linear interpolation of the current image 111.
As the linear interpolation illustrated in FIG. 1B only refers to a single image for all image areas 122, 123, the smooth motion of the text in the ticker window 143 is interrupted and a jerky appearance produced.
After the scene change 120 has taken place, the motion compensated interpolation is continued as illustrated in FIG. 1C.
A scene change 120 detected for a complete image does not automatically also apply to the overlaid ticker window 123. In general, a scene change in the background image 122 will cause a scene change detection signal. However, the appearance of the ticker window, which only represents a small proportion of the total image, has not changed. An improved motion compensated interpolation at scene cuts in accordance with the approach illustrated in FIG. 1B adversely effects the appearance of motion in a ticker window area.
The present invention aims to overcome this drawback and provides an improved method and signal processor for generating new image data.
This is achieved by the features of independent claims.